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Abstract 

A unique and innovative model of primary healthcare (PHC) delivery began at the 
Sudbury District Nurse Practitioner Clinics (SDNPC) in 2007. Significant growth and 
development has taken place since the opening of this new model of PHC delivery. 
Now that SDN PC is operating at full capacity, an evaluation was warranted of patient 
experiences with clinic services and the level of overall patient satisfaction. A survey 
of clinic patients was conducted in the spring of 2011. Results demonstrated high 
satisfaction with clinic services. Some areas were identified for review, including 
the level of accessibility to same-day appointments and rationale for patient use of 
walk-in clinics and emergency department care. One of the strengths and benefits 
of this model of NP care was demonstrated in this evaluation, whereby a majority 
of patients identified that they had received counselling about a lifestyle issue and, 
of those patients, most reported a resulting positive health-related behavioural 
change. Ongoing comprehensive evaluation is important to ensure that services are 
efficient and directed to optimal patient health outcomes. 


Introduction 

The purpose of this evaluation was to explore the experience of patients who 
received healthcare services from the Sudbury District Nurse Practitioner Clinics 
(SDNPC). In particular, the evaluation was focused on determining the level of 
patient satisfaction with accessibility to clinic services and healthcare received at 
the clinic. Participants were also asked if they had made any lifestyle changes as a 
result of counselling during clinic appointments. Finally, participants were invited 
to make recommendations on how the services at SDNPC could be improved. 
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SDNPC opened its doors in 2007, signifying the birth of a unique and innova¬ 
tive model of delivering primary healthcare (PHC). SDNPC was the first NP-led 
clinic in Ontario to receive approval and funding to develop all aspects of govern¬ 
ance and operations related to PHC delivery. Within months of opening SDNPC, 
the premier of Ontario announced the development of 25 additional clinics, 
making the Sudbury model the archetype for future nurse practitioner-led clinics 
(NPLCs) in Ontario (Heale and Butcher 2010). 

The NPLC model was envisioned in response to a significant number of 
“unattached” patients (those without a PHC provider) in the Sudbury area. 
Unattached patients, along with the professional experiences of NPs in the area, 
lobbying from NPs in the community and from professional nursing organiza¬ 
tions, and strong community and governmental support, resulted in the NPLC 
model becoming a reality (Heale and Butcher 2010). An overarching goal of 
SDNPC was to increase accessibility to PHC services as well as to offer high-qual¬ 
ity, comprehensive, evidence-based care that helps patients reach an optimal state 
of health and well-being. 

Individuals without a PHC provider are left with few options when it comes to 
accessing healthcare services. Many attend walk-in clinics or the local hospi¬ 
tal emergency department. Unattached patients do not enjoy the same benefits 
of health promotion, preventive screening, health education and monitoring 
of chronic conditions that are received by those who belong to a PHC centre 
(Howard et al. 2008). The focus of the NPLC model is to register unattached 
patients (DiCenso and Wyman 2008). By taking on unattached patients and 
providing them with comprehensive care, NPLCs are part of the solution of 
healthcare reform and Ontario’s growing PHC challenge (MOHLTC 2011). 

The core of the NPLC model is comprehensive PHC through an interprofessional 
team approach. The concept of an interprofessional team approach is understood as 
one in which the various disciplines in the team interact in a coordinated and mutu¬ 
ally respectful manner, using common terminology, and which includes the patient 
at every step of the decision-making process (Sheehan et al. 2007; Jessup 2007). 

Each patient registered with the clinic was assigned to a NP for the majority of PHC 
needs. As the primary care provider, the NP works with the patient and the members 
of the interprofessional team to ensure that each plan of care incorporates the exper¬ 
tise and feedback of all team members. Additional members of the interprofessional 
team include physicians, a pharmacist, a registered nurse, a social worker and a 
dietitian. This interprofessional team approach is integral to meeting the healthcare 
needs of individuals, their families and their communities (Heale 2012). 
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Evolution of the SDN PC Model 

Tremendous growth and change have taken place in the SDNPC organization 
during its first four years. The mandate of taking on unattached patients has meant 
that people who have registered to become new patients of the clinic had typically 
not been receiving regular PHC services, in some cases for years (Heale 2012). In an 
earlier evaluation of clinic services, it was discovered that a new medical diagnosis 
was discovered for 37% of all new registrants to SDNPC (PRA Inc. 2009). Further, 
the new patients had multiple health problems, which made them more complex to 
manage. As a result of the complexity of the patients registering with SDNPC, the 
process of assessing, diagnosing and stabilizing patients initially slowed the overall 
intake process. Patients with complex health conditions often required ongoing 
assessment over several visits in order to determine their current health status and 
to establish the most appropriate interprofessional treatment plans (Heale 2012). 

In addition to the complexity of its patients, SDNPC did not have fully desig¬ 
nated space nor a full complement of healthcare providers until recently. In an 
attempt to address some of these physical issues, a second site was opened in the 
town of Lively in the summer of 2010, a few kilometres from the original down¬ 
town Sudbury site. The expansion allowed for the hiring of additional healthcare 
professionals, such as a social worker and dietitian, who would ultimately provide 
services to patients at both sites. Many challenges were encountered in the initial 
years. Among these was the delivery of comprehensive PHC services within the 
restrictions of a limited NP scope of practice, high patient complexity, reduced 
space and partial staff complement (Heale 2012). In 2011, changes were made 
to NP regulations through Bill 179, which expanded the NP s scope of practice 
through such means as open prescribing and the addition of controlled acts such 
as dispensing. With the additional space and human resources, as well as much- 
needed legislative changes such as the passing of Bill 179 in fall 2011, SDNPC was 
able to provide care to thousands of people across Greater Sudbury. 

SDNPC has been part of a number of evaluations. Its board of directors conducted 
a patient satisfaction survey after the clinic had been open for a year. The purpose 
of the evaluation was to determine the level of patient satisfaction and identify areas 
of concern or need, including areas for improvement. There was overwhelming 
satisfaction with the clinic as a whole and with NP care in particular. One impor¬ 
tant outcome of this initial assessment was the implementation of extended clinic 
hours one day per week to better accommodate patient schedules (Heale 2008). 

In a previous study conducted in 2009, the Ontario Ministry of Health and Long- 
Term Care requested an evaluation of SDNPC, including a review of the NP model 
of care and patient experience. The 2009 evaluation confirmed a high level of 
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satisfaction with the NPLC model of care. The report identified that patients had 
a good understanding of the NPLC model and that the model worked well. The 
evaluation provided additional data about the NPLC model from the perspective 
of staff and government, and expanded on patients’ experience (PRA Inc. 2009). 

Once both sites became operational and a full staff complement was achieved, the 
need for further evaluation of the clinic model became apparent. The intention 
was not to repeat previous evaluations but rather to consider questions that had 
yet to be addressed related to specific issues. One additional catalyst for this evalu¬ 
ation was information provided by the chief nursing officer of Health Sciences 
North (formerly Sudbury Regional Hospital) during an annual general meeting. 
The hospital identified a decrease in ER visits in 2009 that it attributed to the 
increase in PHC services in the area (D. McNeil, personal communication, 2010). 
Because it was one of the only new PHC initiatives to become available during that 
period, SDNPC was considered to be making a difference, evidenced by an overall 
reduction in patient visits to the local ER. This information helped to inform the 
current SDNPC evaluation, with a stronger focus on access to clinic services. 

Summative evaluations or feedback from patients about their experience as 
members of SDNPC is important (Metcalfe et al. 2008). With SDNPC fully opera¬ 
tional, a process evaluation and outcome assessment were warranted, including 
verification that a program or service was meeting targeted goals (Metcalfe et al. 
2008; Rossi et al. 2004). An overarching goal of SDNPC was to increase accessibil¬ 
ity to PHC services as well as to offer high-quality, comprehensive, evidence-based 
care that helps patients reach an optimal state of health and well-being. With these 
criteria in mind, an evaluation was undertaken to examine the delivery of health¬ 
care services, the extent to which current operations were meeting the needs and 
expectations of the patients and the level of patient satisfaction related to their 
experience with SDNPC. 

Patient Satisfaction 

Establishing a definition of “satisfaction” as it applies to patients and the health¬ 
care system has been difficult. Even so, the concept has been used in healthcare 
research for decades because it offers one measurement of the success in delivery 
of healthcare services (Crow et al. 2002). A systematic review of research related 
to patient satisfaction identified two groups of factors affecting satisfaction: 

(a) the characteristics of the respondents, such as age, health status and health 
outcomes, and (b) health services delivery. There is consistent evidence that the 
most important factor in patient satisfaction is the therapeutic relationship with 
the practitioner (Crowe et al. 2002). In addition, the value that patients assign to 
an aspect of the healthcare system affects their level of satisfaction (Staniszewska 
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and Ahmed 1999). For example, Penchansky and Thomas (1981) determined that 
there is a link between accessibility to healthcare services and level of satisfaction 
with the healthcare system. 

The patient satisfaction study strove to identify basic demographic characteristics 
of the participants and their responses with respect to key issues about health 
services delivery at SDNPC, such as accessibility of appointments and patients’ 
interaction with their primary NP healthcare provider. The demographic charac¬ 
teristics were compared to the participants’ rating of level of satisfaction with the 
SDNPC. A snapshot of patient experiences and overall impression of the SDNPC 
model was thereby achieved. 

Methodology 

Evaluation of patient experiences with clinic services and their overall level of 
satisfaction was conducted through a survey. The survey questions were devel¬ 
oped based on previous clinic evaluations, a literature review of related concepts, 
the professional experience of the researchers and anecdotal feedback from 
patients. Basic demographic information, such as gender, age and employment 
status, was collected during the course of the evaluation. Questions in the patient 
survey addressed accessibility and included the length of time to obtain a routine 
appointment and whether same-day appointments were available. Another goal 
of the survey was to determine which options for PHC services were taken by the 
patient if a clinic appointment wasn’t available when they felt it was required. 
Specifically, we wanted to determine to what extent SDNPC patients were access¬ 
ing other services, such as local ER departments or walk-in clinics. Questions relat¬ 
ing to scheduling and alternatives were included. Patients were asked to respond 
to questions specific to their experience over the previous six months, a period 
that corresponded with the clinic’s attaining a full staff complement and optimum 
space. Respondents were asked questions about lifestyle changes that they had 
made as a result of their encounters with NP-focused care. Einally, patients were 
asked to rate their overall satisfaction with the clinic on a five-point Likert scale 
and were given the opportunity to include comments about clinic services. 

A convenience sample of 1,865 people was obtained from the SDNPC patient 
database. Those eligible for the survey were over the age of 18, had registered as 
patients at SDNPC and had a home address in the Sudbury region. Each eligible 
person was mailed a survey package, which included a copy of the survey and 
cover letter in English and Erench, as well as a stamped return envelope. The study 
was approved by SDNPC’s board of directors as well as Laurentian University’s 
Research Ethics Board. 
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Results 

Analysis of the survey responses was conducted in several ways. Data from the 
survey questions were coded and entered into a statistical program file. Frequency 
statistics provided the basis of analysis for the demographic data and checked- 
off responses to questions. Responses to open-ended questions were analyzed by 
descriptive analysis using coding and identification of major themes from the 
comments provided. 

Of the 1,865 surveys that were mailed, a total of 682 surveys were returned within 
a period of 3 months. This translated into a 36.5% response rate. Of these, 435 
(63.7%) respondents were female and 230 (33.7%) were male. There were 17 
(2.5%) respondents who did not identify their gender. (The age range of the 
respondents is presented in Table 1.) 


Table 1. 


Age ranges of survey participants 



Frequency 

Percentage 

18-29 

74 

10.9 

30-39 

92 

13.5 

40-49 

105 

15.4 

50-59 

172 

25.2 

60-69 

124 

18.2 

70+ 

109 

16.0 

Missing 

6 

0.9 

Total completed surveys 

682 

100.0 


Twelve (1.8%) of the surveys were completed in French. Respondents were asked 
how long they had been registered at SDNPC. The majority, 52.8% (n=360), had 
been registered with the clinic for more than two years, followed by 26.5% (n=181) 
for one to two years and 19.8% (n=l 11) who had been registered for less than a year. 

Respondents were asked to state, on average, the length of time it took for them to 
obtain a routine appointment. The largest number of respondents (249 or 36.5%) 
chose the “1-2 weeks” category. Of the remaining respondents, 112 (16.4%) chose 
“less than 1 week”; 144 (21.1%) chose “2-3 weeks”; and 101 (14.8%) chose “more 
than 3 weeks.” Another 63 (9.2%) chose “not applicable,” and 13 did not respond 
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to the question. These 76 respondents likely did not require an appointment over 
the last six months. 

Patients were asked to respond to several short-answer questions. Three hundred 
sixteen indicated that they had received counselling about a lifestyle issue. Of these, 
the majority identified that they had quit or reduced smoking, or that they had 
made other positive changes to their lifestyle, such as improved diet or increased 
exercise. Many respondents identified the effects of their targeted lifestyle changes 
including weight loss, lowered BP and improved management of chronic disease. 
Still others identified positive changes in stress management, compliance with 
treatment plans or implementation of prevention strategies such as immunizations. 

At the end of the survey, the participants were simply asked how they felt SDNPC 
services could be improved. Responses to this question fell into several categories. 
The majority of respondents identified issues related to access, such as booking 
of appointments, including increasing the availability of same-day appointments, 
decreasing the time to obtain an appointment and considering extended hours. 
Some of the respondents demonstrated a lack of understanding of the interprofes¬ 
sional clinic model in identifying the need to see their “doctor.” At the same time, 
many participants identified the need for a second appointment with the clinic 
physician to have medications renewed because of the limitations imposed by the 
NP drug list during the evaluation timeframe. 

Accessibility was addressed in several “yes/no” questions. Table 2 lists these and 
other survey questions, along with the frequency and percentages of responses. 
Issues related to wait times to get an appointment and accessing other forms of 
PHC other than those offered by the SDNPC were also explored. For those who 
felt they required a same-day appointment, 38.9% (265) felt that they had not 
been able to obtain one, while 19.8% of those who felt a same-day appointment 
was needed were successful. With respect to the utilization of other services, 

38.1% (260) of the respondents accessed a walk-in clinic, 24.6% (168) visited an 
ER department and 10.3% (70) contacted Telehealth during the last six months. 
Of these, 11.3% (77) required an admission to hospital. 

Participants were asked to rate their level of satisfaction with the clinic on a five- 
point Likert scale. The majority of the respondents were either satisfied or very 
satisfied with the services that they received over the last six months (Figure 1). 

The level of patient satisfaction was compared to several variables using a chi-square 
analysis. Higher levels of satisfaction were significantly associated with access, 
such as shorter wait times to schedule a routine appointment (p=.000), the abil¬ 
ity to obtain a same-day appointment if needed (p=.000) and waiting less than 15 
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minutes from time of scheduled appointment to see a healthcare provider (p=.000). 
In addition, satisfaction with the clinic was significantly higher in those who did not 
attend a walk-in clinic or emergency department (p=.002). There were no signifi¬ 
cant differences in satisfaction between patients who were admitted to hospital and 
those who were not, or between those who called Telehealth and those who did not. 


Table 2. 


Responses to survey questions 



Yes 

No 

N/A 

Missing 

1 have been able to see a nurse practitioner for a same- 
day appointment, if needed. 

135 

(19.8%) 

265 

(38.9%) 

268 

(39.3%) 

14 

Since joining the clinic, 1 have never had to wait longer 
than 15 minutes past my scheduled appointment time 
to see a healthcare provider. 

447 

(65.5%) 

216 

(31.7%) 

10 

(1.5%) 

9 

1 have a medical condition(s) that requires regular 
medical appointments at the clinic. 

227 

(33.3%) 

376 

( 55 - 1 %) 

69 

(10.1%) 

10 

Since joining the clinic, my medical condition(s) is 
under better control. 

309 

(45.3%) 

51 

(7.5%) 

311 

(45.6%) 

11 

1 have received counselling from a member of the 
healthcare team about a lifestyle issue (e.g., food 
choices, exercise, smoking cessation, etc.). 

316 

(46.3%) 

184 

(27.0%) 

175 

(25.9%) 

7 

1 have made improvements to my lifestyle as a result of 
the counselling received by the clinic staff (e.g., started 
exercising, decreased or stopped smoking, etc.). 

251 

(36.8%) 

121 

( 17 - 7 %) 

274 

(40.2%) 

36 

Since joining the clinic, 1 have been to a walk-in clinic for 
a health issue for me, or for a child in my family. 

260 

(38.1%) 

391 

( 57 - 3 %) 

2 

(0.3%) 

29 

Since joining the clinic, 1 have been to a hospital 
emergency department for a health issue for me, or for 
a child in my family. 

168 

(24.6%) 

486 

(71.3%) 

654 

(95.9%) 

28 

Since joining the clinic, 1 have been admitted to 
hospital for a medical condition (other than childbirth). 

77 

(11.3%) 

575 

(84.3%) 


30 

Since joining the clinic, 1 have called Telehealth for a 
health issue for me, or for a child in my family. 

70 

(10.3%) 

582 

(85.3%) 

1 

(0.1%) 

29 


There was no significant difference in satisfaction with the clinic based on gender. 
There was, however, a difference in level of satisfaction based on age, whereby the 
youngest group of patients tended to be less satisfied with the clinic services in 
particular when it came to accessing same-day appointments. This was also noted 
in the group aged 70 and over who expressed lower levels of satisfaction than 
middle-aged counterparts. 
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Levels of satisfaction 


400 - 



very unsatisfied unsatisfied neither satisfied satisfied very satisfied 

nor unsatisfied 



Patients who indicated that they had achieved better control of a medical condi¬ 
tion since joining the clinic were significantly more satisfied with the clinic 
(p=.00). In addition, patients who received counselling about a lifestyle issue (e.g., 
food choices, exercise, smoking cessation, etc.) also tended to be more satisfied 
with the clinic (p=.000). 


Discussion 

The goal of the evaluation was to determine patients’ experience with 
SDNPC services and their overall level of satisfaction. The results outlined 
areas of success in healthcare delivery, as well as areas for improvement. 
What is clear is that there appears to be a high level of satisfaction among 
patients of the SDNPC who participated in this survey. 

The level of satisfaction is increased with accessibility to routine appoint¬ 
ments and the ability to schedule same-day appointments if required. 
Satisfaction decreases when patients seek care in walk-in clinics or the 
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ER. Presumably, many ER visits were warranted, given that 11.3% of the 
participants indicated that they were hospitalized during the review period. 
However, the level of acuity or appropriateness of the care sought in walk-in 
clinics and the ER was not reviewed, and a comparison with the patterns of 
use by the general population is not known. Therefore, the rationale for and 
appropriateness of the use of these services is not known. 

Several strategies could be implemented within the clinic operations to 
address appointment delays and patterns of seeking care from external 
sources. One option is the implementation of advanced access schedul¬ 
ing. Advanced access is a system for scheduling appointments that opens 
up a large percentage of same-day appointments without the requirement 
of additional healthcare providers and staff (Eournier et al. 2012). Another 
option is a review of expanded hours, which may better meet the needs of 
working parents (Howard et al. 2008). Work to evaluate and strengthen the 
interprofessional team approach may increase patient accessibility to clinic 
services (McCallin 2001; Sheehan et al. 2007). Einally, because many patients 
did not have a PHC provider prior to registering with the clinic, the only 
source of ongoing care was likely walk-in clinics and the ER. Patient aware¬ 
ness of the clinic model is important to encourage appropriate use of walk- 
in clinics and the emergency department as alternatives to care. 

A higher level of satisfaction is a reflection of the care provided at the clinic, 
such as better control of medical conditions and lifestyle counselling. The 
large percentage of respondents who identified that they had received life¬ 
style counselling is congruent with NP surveys indicating that 23% of NP 
practice is health promotion (CRaNHR 2008). The positive lifestyle changes 
that were made as a result of counselling received by SDNPC patients reflects 
the value of a model of care that is holistic and interprofessional, and that 
integrates the patient in decisions about care. However, the health outcomes 
that were identified by the survey respondents were not confirmed with 
clinic data. An added qualitative analysis and confirmation of health status 
in patient charts would have provided richer data about patients’ experience 
with the clinic (Davies et al. 2010). 

There was less satisfaction among the older age group than among those in 
the middle-age range. What is not known from this study is whether there 
is an association between age and the level of complexity, or acuity of health 
conditions, in the older sample. Future research should target the evaluation 
of appropriate patient load for NPs in NPLCs based on the complexity and 
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level of acuity of the patient s health problems and subsequent accessibility 
to appointments and patient satisfaction with healthcare service. 

NP regulation affects the delivery of care and, ultimately, patient satisfac¬ 
tion. In response to the question about how service at SDNPC could be 
improved, patients identified the need to see the physician for renewal of 
some prescriptions as a source of frustration. At the time that the survey 
was conducted, NPs in the province of Ontario were restricted to follow¬ 
ing established drug lists from which to prescribe. The list had long been 
identified as a significant barrier to NP practice, and one that contributed 
to some of the patient dissatisfaction related to access that was expressed in 
the survey (HPRAC 2009). Fortunately, on October 1,2011, the College of 
Nurses of Ontario, in response to the implementation of Bill 179, established 
revised Standards of Practice for Nurse Practitioners (CNO 2011). One 
element is the removal of the list of drugs from which NPs could prescribe. 
Although controlled substances are still outside NP prescriptive authority, 
the legislative changes will create a more streamlined and efficient care path 
for patients, because NPs no longer have to consult with physicians for such 
things as simple medication renewals. 

Sudbury is a northern community with a medium-sized population, which 
may not be representative of larger urban settings or more rural settings. For 
the most part, SDNPC patient demographics and past experiences with the 
healthcare system are unique to this geographic area. The same evaluation, 
performed at various NPLCs across Ontario, would offer insights into the 
NPLC model, rather than focus on one organization. 

Conclusions 

In order to identify what is effective and what requires improvement, there 
is a need for organizations to conduct ongoing and frequent evaluations of 
the services they deliver to their patients (Hills and Carroll 2012). Although 
patient satisfaction with the SDNPC model is high, past evaluations, in addi¬ 
tion to the recent survey results, have demonstrated that there is room for 
improvement. Timely access remains an important criterion for consumers 
of PHC services throughout the healthcare system, and Sudbury is no excep¬ 
tion (Howard et al. 2008). Extended clinic hours and the potential for open 
access scheduling is an option the SDNPC can accommodate, which would 
demonstrate that patient satisfaction is important and that NP-led clinics 
would continue to remain leaders and role models for PHC delivery. Given 
the high level of complexity of many of the patients registered at SDNPC, an 
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evaluation of appropriate patient-provider caseloads is warranted. By deter¬ 
mining the ideal number of patients that each NP should oversee, the clinic 
may be able to further address the issue of increasing access to appointments. 

Research to identify patient understanding and perceptions about clinic 
accessibility is warranted. Also, a review of clinic data may be used to 
confirm patient health outcomes. The unique features of an NP-led, inter¬ 
professional model of care are worthy of further exploration, and could be 
realized with a comprehensive evaluation strategy that includes all opera¬ 
tional NPLCs in Ontario. 

The NP-led clinic model has proven to be a successful strategy in address¬ 
ing the large number of unattached patients in Ontario. The responses to the 
survey suggest that as a result of accessing PHC services at SDNPC, partici¬ 
pants have made positive lifestyle changes such as quitting smoking, eating 
better or becoming more active. Ongoing evaluation confirms that the patients 
value the NP-led clinic model of care. Continued review will provide additional 
evidence about the effectiveness of the model in ensuring optimal health. 
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